System and method for counting follicular units

ABSTRACT

A system and method for counting follicular units using an automated system comprises acquiring an image of a body surface having skin and follicular units, filtering the image to remove skin components in the image, processing the resulted image to segment it, and filtering noise to eliminate all elements other than hair follicles of interest so that hair follicles in an area of interest can be counted. The system may comprise an image acquisition device and an image processor for performing the method. In another aspect, the system and method also classifies the follicular units based on the number of hairs in the follicular unit.

FIELD OF INVENTION

This invention relates generally to hair transplantation procedures andmore particularly to a system and method for counting follicular unitsusing digital imaging and processing techniques for use in hairtransplantation procedures.

BACKGROUND

Hair transplantation procedures are well-known, and typically involve(in a patient having male pattern baldness) harvesting donor hair graftsfrom the side and back fringe areas (donor areas) of the patient'sscalp, and implanting them in a bald area (recipient area).Historically, the harvested grafts were relatively large (3-5 mm),although more recently, the donor grafts may be single follicular units.In particular, “follicular units” (also referred to herein as FU or FUs)are naturally occurring aggregates of 1-3 (and much less commonly, 4-5)closely spaced hair follicles that are distributed randomly over thesurface of the scalp.

The follicular units may be classified, or “typed,” based on the numberof hairs in the unit and identified in shorthand as an “F1” for a singlehair follicular unit, an “F2” for a two hair follicular unit and so onfor follicular units with 3-5 hairs. In some cases of multiple hairfollicular units, the hairs may appear to emanate from a single follicleor point in the skin. In other cases, the hairs may exit the skinsurface at slightly spaced apart positions, but converge into a singlefollicular unit beneath the skin. Referring to FIG. 1, a print of adigital image of an exemplary section of a human scalp 11 having avariety of types of follicular units is shown. For example, thefollicular unit 17 has two hairs and is therefore an F2, whilefollicular unit 13 is an F1 since it has only a single hair.

There are several reasons it is important and desirable to count andclassify follicular units in a region of interest on a body surface. Forone, the number of follicular units can be used in the planning processfor a transplantation procedure. For instance, if this number sets thelimit on the number of follicular units in that area that can beharvested for transplantation. However, in many cases, the doctor maywant to implant only a certain percentage of the follicular unitsavailable, thereby leaving some coverage in the area being harvested. Inaddition, in many hair restoration transplant procedures, certainclasses of follicular units are preferred.

As for classification, there are several reasons it is important anddesirable to identify and classify follicular units based on the numberof hairs in the follicular unit. It may be desirable to utilize avariety of classes (also referred to as “types”) of follicular units toprovide the desired attributes for the appearance of the transplantedhair. Such attributes can include the density of hair, the direction ororientation of hair, the particular mix of types of follicular units,and/or the appearance of randomness, among other possible attributes. Anexample of the use of various types of follicular units is as follows.It is preferable to transplant certain classes of follicular units intospecific regions of the scalp. For example, single hair follicular units(F1s) are commonly implanted along the hairline that frames the face.Follicular units with more than one hair (F2s, F3s, etc.) are commonlyimplanted in the mid-scalp and crown. This arrangement of follicularunit distribution is thought to produce a more natural appearingaesthetic result.

Various procedures for hair transplantation have been previouslydisclosed, including both manual and mechanized to certain degrees ofautomation. In one well-known manual process, a linear portion of thescalp is removed from a donor area by dissection with a scalpel downinto the fatty subcutaneous tissue. The strip is dissected (under amicroscope) into the component follicular units, which are thenimplanted into a recipient area in respective puncture holes made by aneedle. Forceps are typically used to grasp and place the follicularunit grafts into the needle puncture locations, although otherinstruments and methods are known for doing so.

In “Androgenetic Alopecia” (Springer 1996), M. Inaba & Y. Inaba discloseand describe a manual method for harvesting singular follicular units bypositioning a hollow punch needle having a cutting edge and interiorlumen with a diameter of 1 mm, which is about equal to the diameter ofcritical anatomical parts of a follicular unit. The needle punch isaxially aligned with an axis of a follicular unit to be extracted andthen advanced into the scalp to cut the scalp about the circumference ofthe selected follicular unit. Thereafter, the follicular units areeasily removed, e.g., using forceps, for subsequent implantation into arecipient site with a specially devised insertion needle.

U.S. Pat. No. 6,585,746 discloses an automated hair transplantationsystem utilizing a robot, including a robotic arm and a hair follicleintroducer associated with the robotic arm. A video system is used toproduce a three-dimensional virtual image of the patient's scalp, whichis used to plan the scalp locations that are to receive hair graftsimplanted by the follicle introducer under the control of the roboticarm. The entire disclosure of U.S. Pat. No. 6,585,746 is incorporatedherein by reference.

Automated systems and methods for transplanting are also disclosed inU.S. provisional patent application Ser. Nos. 60/722,521, filed Sep. 30,2005, 60/753,602, filed Dec. 22, 2005, and 60/764,173, filed Jan. 31,2006, and U.S. patent application Ser. Nos. 11/380,903, filed Apr. 28,2006 (now published as US 2007/0078466) and 11/380,907, filed Apr. 28,2006 (now published as US 2007/0106306). The foregoing applications areall hereby incorporated by reference into the present application intheir entirety. For example, in U.S. patent application Ser. No.11/380,907, referenced above, the disclosed system comprises a roboticarm having a harvesting and/or implantation tool mounted on the arm. Oneor more cameras are also mounted on the arm and are used to image thework space, such as a body surface. A processor is configured to receiveand process images acquired by the cameras. A controller is operativelycoupled to the processor and the robotic arm. The controller controlsthe movement of the robotic arm based, at least in part, on theprocessed images acquired by the cameras and the processor. The arm iscontrollably moveable to position the tool at a desired orientation andposition relative to the body surface to perform transplantation ofhairs.

In utilizing any of these systems and methods for hair transplantation,it is desirable to first plan the transplantation to select thefollicular units to be harvested and transplanted and to determine theprecise location where the hairs are to be implanted. Accordingly, inplanning a hair transplantation procedure, specific follicular unitsfrom a specific location on a body surface may be selected forharvesting and transplantation into a different part of the bodysurface. The follicular units to be transplanted may be selected basedon certain criteria, for example, the type of follicular unit (i.e. F1,F2, etc.), the orientation of the hair in the follicular unit, thedensity of the hair, etc. However, the process of counting, andcharacterizing each follicular unit can be tedious and time consuming.Therefore, there is a need for a system and method for counting and/orclassifying follicular units using an automated system. A system andmethod for classifying follicular units is described in U.S. patentapplication Ser. No. 11,467,268, filed on or about Aug. 25, 2006,entitled SYSTEM AND METHOD FOR CLASSIFYING FOLLICULAR UNITS, thecontents of which are incorporated by reference herein in its entirety.

SUMMARY

In accordance with a general aspect of the inventions disclosed herein,a system and method for counting follicular units using an automatedsystem is provided. The system and method of the present invention maybe utilized with systems and methods for transplantation of hairfollicular units on a body surface. The system and method of the presentinvention is especially useful when implemented on, or integrated with,an automated system for hair transplantation.

In one aspect of the present invention, the method of countingfollicular units comprises acquiring an image of a body surface havingskin and follicular units, filtering the image to remove backgroundcomponents (such as skin, and optionally, some other small backgroundcomponents) in the image, processing the resulted image to produce asegmented image, and filtering noise to remove objects that do notcorrespond to the follicular units of interest. As a result, theremaining follicular units of interest can be counted. In one preferredembodiment, the acquired image is a digital image, although it is notnecessary and analog images may be used. The analog image may beconverted into a digital image using techniques known to those ofordinary skill in the art. According to the present invention, the stepof removing skin components (and/or other background components) in theacquired image may be accomplished without limitation by any appropriatetechniques and methods.

In one exemplary embodiment, filtering to remove background componentsis accomplished using a background subtraction technique where the skinsurface is the background. One of the exemplary background subtractiontechniques includes subtracting a leveled image (or blurred version ofthe input image) from the input image. Others involve a variance filter,edge detection based techniques, or keying off skin tones using colordata. In another exemplary embodiment the step of removing background(skin) components is accomplished using a band-pass filter. Processingthe image to produce a segmented image may be accomplished by anywell-known techniques. In one exemplary embodiment of the method of thepresent invention, the segmented image is a binary image; however,multi-modal images (e.g. skin, moles, blood and/or other features arerepresented by more than two different image codes) are also within thescope of the present invention. The segmented image is further subjectedto noise filtering, as necessary, to remove everything but hairfollicles of interest. Examples of the type of noise that needs to befiltered include, but are not limited to, image noise, dandruff, bloodspeckles, moles, long uncut hair, etc. All noise filtering could beaccomplished simultaneously, or it could be broken into several separatesteps: for example, first “small noise” corresponding to the smallobjects (e.g. small dandruff) is filtered, and then “larger noise”corresponding to the larger objects (e.g. long hair, large speckles ofblood) is filtered.

Filtering of this “small noise” by removing such objects from thesegmented image is referred to as a morphological open operation, whichis a standard image processing technique known by those of ordinaryskill in the art. The rest of the noise filtering is then performed onthe image resulting from the morphological open operation. The noisefiltering removes objects which do not meet criteria corresponding to afollicular unit. For example, the area, location or orientation of anobject in the image may be one whose area, location or orientation doesnot correspond to an actual follicular unit (it could be cut hair thathappens to be remaining on the scalp, for example). Whether thecharacteristics of an image of an object corresponds to hair may bedetermined by statistical comparison to the global nature of the samecharacteristics for images of objects in the selected image which areknown to be hair, or alternatively, the characteristics can be comparedto predetermined criteria based on patient sampling or other data (e.g.,if the patient parts the hair in a certain way we know that the hairsshould mostly be pointing in a given direction).

Each of the objects remaining in the image after the noise filtering iscounted as a follicular unit. Thus, the method may be used to countfollicular units.

In another aspect of the method of counting follicular units, filteringto remove skin components of the image using a band-pass filter maycomprise a first filtering step using a low-pass filter having a firstkernel and a second filtering step using a low-pass filter having asecond kernel. In another feature of the present invention, the low-passkernels may be Gaussian kernels. Those of ordinary skill in the art arefamiliar with, and understand how to implement, such low-pass filtersand Gaussian filters.

In yet another embodiment of the method of the present invention, thecounting of follicular units may be refined by using, for example,multiple imaging. It may also include a method for tracking the FU ofinterest and aligning the system to obtain the image. In one exemplaryembodiment, first and second cameras are used to provide stereo images.The stereo images may be used to track an FU of interest within theimages of the first and second cameras to adjust for movement of thebody surface and/or movement of the cameras. In addition, the first andsecond cameras are aligned with the general orientation of the hair ofthe FU, so that images obtained provide good quality data for performingthe remaining steps of the method of the present invention. The stereoimages or multiple images may also be used to compute coordinatepositions of the hairs. Then, images having a computed coordinateposition which is inconsistent with a hair on said body surface can alsobe filtered out. Alternatively, the system and method of the presentinvention can use multiple cameras (or other image acquisition devices)or only one camera to make multiple images from various angles,including panoramic images. Camera could be moved either manually orwith the assistance of a robot if the system used is a robotic system.This optional count refining step could be used as needed.

In another embodiment of the present invention, the method of countingof follicular units may be used in conjunction with a method ofclassifying follicular units, such as the methods described in U.S.patent application Ser. No. 11/467,268. In this way the follicular unitsof interest may be both counted and classified. The method ofclassifying a follicular unit (FU), as described in the Ser. No.11/467,268, comprises acquiring an image of a body surface in whichthere are follicular units (FU) and processing such image to produce asegmented image of the FU. In one preferred embodiment the segmentedimage is a binary image, but it could be a multi-modal image, asdescribed above. From the segmented image of the FU, a contour aroundthe outer perimeter of the hair(s) of the FU may be calculated. Forexample, for an F1, the contour would generally be a line or surfacefollowing the outer surface of the single hair. The segmented image alsoallows the calculation of an outline profile of the FU. The outlineprofile disregards concavities in the contour of the image.

The outline profile is then compared to the contour to determine thenumber of “defects” in the outline profile. A defect in the outlineprofile may be defined, for example, as each of the concavities in theoutline profile which divert from the contour. In the F2 example, thereis one defect in the outline profile represented by the concavity formedby the “V” shape. In an F3, the contour will be generally shaped liketwo Vs sharing a common vertex and with one line forming one side ofboth Vs. The outline profile of an F3 will also have a generallytriangular shape (although it may be a wider triangle than an F2). Thus,an F3 will have two defects. Therefore, it can be seen that the numberof defects has a direct relationship to the type of follicular unit. Inthis case, the number of hairs for the FU equals the number of defectsminus one.

In one embodiment of the method of classifying follicular units, theoutline profile may be determined by calculating a convex hull contourpursuant to well-known image processing techniques. Other appropriatetechniques for determining the outline profile are also within the scopeof the invention disclosed.

In another embodiment of the method of classifying follicular units,procedure is provided for tracking the FU of interest to adjust forrelative movement between an image acquisition device and the FU.Multiple image acquisition devices, such as cameras, may be aligned toobtain an image. In one exemplary embodiment, first and second camerasprovide stereo images. The stereo images may be used to track an FU ofinterest within the images of the first and second cameras to adjust formovement of the body surface and/or movement of the cameras. Inaddition, the first and second cameras are aligned with the generalorientation of the hair of the FU. In this way, an image is obtainedwhich provides good quality data for performing the remaining steps ofthe method of classifying the FU.

In addition, the method of classifying a follicular unit may also adjustfor follicular units having hairs which converge below the surface ofthe skin. In such case, the image will contain an image of a hair whichis not a contiguous part of the contour of the FU of interest. Toaccount for this situation, it is determined whether the separate hairis within a maximum distance from the hair(s) defining the contiguouscontour of the FU of interest. The maximum distance is set to be adistance in which what appears to be a hair from a separate FU is mostlikely a part of the same FU as the FU of interest. The classificationof the FU of interest then takes into account any additional hair(s)which are within a maximum distance from the hair(s) of the FU ofinterest.

Furthermore, the method of classifying a follicular unit may also adjustfor hair images which appear to be a single hair but are in actualitymultiple hairs. Thus, determining the number of defects will not providean accurate classification because the merged hairs will result in fewerdefects in the outline profile (and therefore fewer hairs) than areactually present in the FU of interest. To account for this situation,the method determines the width (or caliber) of each object representinga hair in the FU of interest using the image. Then, it is determinedwhether the width of each object representing a hair exceeds a maximumexpected width for a single hair and compares them. The step ofclassifying the FU may also be based on a result of the above comparisonand determination whether the width of an object representing a hairexceeds the maximum expected width and by how much. For example, if thewidth is between 1½ and 2 times the expected width, then the step ofclassifying will approximate such object as being two hairs. A similarapproximation can be done for 3, 4 or 5 hairs.

Where the method of counting FUs is used in conjunction with the methodof classifying an FU, the method of counting may be performed before,after or simultaneously with the method of classifying.

In another aspect of the present invention, a system for counting (andin some embodiments, also classifying) follicular units is provided. Inone exemplary embodiment of the present invention, the system forcounting an FU using an automated system comprises an image acquisitiondevice and an image processor. One example of the image acquisitiondevice is one or more cameras, such as any commercially availablecameras. Instead of a camera, it could be a video recording device (suchas a camcorder) or any other image acquisition device. While stereoimaging devices work well with the present invention, it is notnecessary to have stereo imaging. Similarly, while it is preferred thatthe image acquisition device be a digital device, it is not necessary.It could be, for example, an analog TV camera that acquires an initialimage which may be then digitized for further use in the method of thepresent invention. The image processor may comprise any deviceprogrammed and configured to perform the method of counting (and,optionally classifying) an FU according to the present invention. Onenon-limiting example of a suitable image processor is any type ofpersonal computer (“PC”). Alternatively, the image processor maycomprise an Application Specific Integrated Circuit (ASIC) or FieldProgrammable Gate Array (FPGA).

According to another aspect of the present invention, the imageprocessor is provided that is programmed and configured to perform themethod of counting (and, optionally classifying) an FU according to thepresent invention. Any suitable image processor is within the scope ofthe present invention. In one exemplary embodiment, an image processorfor counting follicular units is configured for receiving an image of abody surface comprising skin and follicular units, filtering the imageto remove background components, processing the image to produce asegmented image, and performing noise filtering to remove objects thatdo not correspond to follicular units of interest. Such image processoraccording to the present invention could be used in conjunction withvarious systems for planning of hair treatments, for harvesting and/orimplanting follicular units (manual, semiautomatic, automatic, orrobotic), as well as with various systems for counting or classifyingfollicular units; alternatively, it could be incorporated in any of suchsystems.

A system for counting follicular units using an automated system may beused in conjunction with or may comprise any of the transplantationsystems described in the background above. For instance, the systemdescribed in U.S. patent application Ser. No. 11/380,907 may beprogrammed and configured to perform the methods of counting follicularunits according to the present invention. The cameras on the system canprovide stereo digital images and the robotic arm can properly positionand orient the cameras. The selection of a region of interest may beperformed by an operator at the user interface of the system (such as acomputer having a monitor and input devices) or it could be automatedthrough programming of the computer and/or controller.

Accordingly, a system and method for counting (and in some embodiments,also classifying) follicular units is provided. Other and furtherembodiments, objects and advantages of the invention will becomeapparent from the following detailed description when read in view ofthe accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is illustrated by way of example and not limitation in thefigures of the accompanying drawings, in which like references indicatesimilar elements, and in which:

FIG. 1 is a print of a digital image of an exemplary section of a humanscalp having a plurality of follicular units.

FIG. 2 is a print of the digital image of FIG. 1 after it has beenfiltered to remove skin components.

FIG. 3 is a print of the digital image of FIG. 2 after the image hasbeen segmented.

FIG. 4 is print of the digital image of FIG. 3 after a morphologicalopen operation has been performed on the segmented image.

FIG. 5 is a print of the digital image of FIG. 4 after all noisefiltering has been performed on the image.

FIG. 6 is an exemplary flow chart of the method of counting offollicular units according to one exemplary embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS

Referring first to FIG. 1, the system and method for counting follicularunits according to the present invention generally begins with acquiringan image 10 of a body surface 11 using any appropriate image acquisitiondevice. In one exemplary embodiment, such image acquisition device isone or more digital cameras. Alternatively, any other suitable imagingdevice may be utilized. The image acquisition device may produce adigital image, such as that produced by a digital camera, or it mayproduce an analog image (which may or may not be converted to a digitalimage at any point in the process). Although in this description of anexemplary embodiment, the image 10 is a digital image taken by a digitalcamera, the present invention is not limited to digital images taken bydigital cameras. Instead of a camera, it could be a video recordingdevice (such as a camcorder) or any other image acquisition device.While stereo imaging devices are currently preferred, it is notnecessary to have stereo imaging. Similarly, while it is preferred thatthe image acquisition device be a digital device, it is not necessary.It could be, for example, an analog TV camera that acquires an initialimage which is then processed into a digital image for further use inthe method of the present invention. The body surface 11 has skin 12 anda plurality of follicular units 14 each having one or more hairs 13(only a few of the follicular units 14 and hairs 13 are labeled in thefigures). The photo of FIG. 1 is an image of a section of human scalp11, but it is understood that the body surface 11 could be any area ofany body having hair. The digital image 10 shows a variety of types offollicular units 14 (FU) on the scalp 11.

The image 10 may be acquired using one or more cameras of an automatedhair transplantation system, such as the cameras described in the hairtransplantation system of U.S. patent application Ser. No. 11/380,907,which is incorporated by reference herein in its entirety. The imagefrom just one of the cameras can be used to produce the digital image10. Alternatively, the process for obtaining the digital image 10 may beacquired by a more involved process which aligns the camera(s) toimprove the image used to count a follicular unit of interest. In thisexemplary process, a first camera and a second camera are used. Thecameras are arranged and configured to obtain stereo images of a bodysurface at which cameras are directed. The cameras are first positionedto be directed at the body surface in an area known to have hair. Afirst image is acquired from the first camera and a follicular unit (FU)of interest is selected from within the first image. A second image ofabout the same region of the body surface as the first camera (exceptfrom a slightly different angle as provided by stereo cameras) isacquired from the second camera and the same FU of interest is selectedfrom within the second image. The FU of interest can be selected in theimages by an operator of the system or automatically by the system usinga selection algorithm. The transplantation system is now able to trackthe FU of interest within the first and second images from the first andsecond cameras. The tracking procedure can be used to adjust formovement of the body surface and movement of the cameras when they arealigned to acquire the image(s) used for counting the FU.

Next, the first and second cameras are moved and oriented to be alignedwith the general orientation of the hair of the FU. As the cameras aremoved, additional images may be acquired and processed by the system inorder to track the FU of interest. By aligning the cameras with the hairof the FU, a better image for counting the FU can be acquired. With thecameras in the desired alignment, the cameras acquire the images to beused in the next steps of the method of counting follicular units. Theabove description is provided strictly by way of example and notlimitation. It is possible to use various multiple image acquisitiondevices, such as multiple cameras, or a single camera to take multipleimages, including panoramic images, at different angles to track the FUof interest. As was previously explained, the image acquisition devicedoes not have to be a digital image acquisition device, and the imagesinitially obtained do not need to be digital images. The movement of thecamera could be controlled by a robotic system, or manually, dependingon the kind of system used.

When the image 10 is acquired, a region of interest 19 could be theentire image 10 or a selected sub-area of the entire image 10. In theexample described herein, the selected region of interest 19 isco-extensive with the image 10. However, the selected region of interest19 can be any subset area of the image 10. The region of interest 19 maybe selected by an operator or the selection may be automated by thesystem. This region of interest within the image may be called theselected image 19. Accordingly, further references to a region ofinterest 19 or selected image 19 may refer to the entire image 10, or toany sub-area which may be selected or simply an inherent result of theacquired image 10.

It has been determined that it is beneficial first to remove backgroundcomponents from the acquired image in order to improve the accuracy andefficiency in the overall process for counting follicular units ofinterest. Generally, such background components correspond to the skin.In addition to the skin, these background components may also includesome additional small objects, for example, dandruff or small bloodspeckles. Accordingly, further references to the filtering or removal ofthe skin (or skin components) is not limited to the skin alone butrather may include some additional small background components orobjects; and any such references should mean filtering or removal of thebackground components. Any suitable method(s) for filtering backgroundcomponents, such as the skin, from the image are within the scope of thepresent invention.

In one exemplary embodiment, filtering of the skin 12 is accomplishedusing a background subtraction technique where the skin surface is thebackground. One of the exemplary background subtraction techniquesincludes subtracting a leveled image (or blurred version of the inputimage) from the input image. A correction may be made for non-uniformillumination and skin tones simultaneously by subtracting a blurredversion of the input image from the input image. The blurred image maybe formed by computing the local means of the selected image 19 (inputimage). The local means are calculated efficiently by convolving theinput image with a Gaussian kernel. This filtering procedure may betuned for a particular patient, ambient lighting, or other clinicalsituation, by adjusting the characteristics of the Gaussian kernelemployed during the blurring process. In summary, the backgroundsubtraction technique of the above-described example is as follows:

(1) Blurred Image=Input image convolved with Gaussian (note: Gaussiancould be replaced by any other suitable kernel)

(2) Image without Background=(Input Image)−(Blurred Image)

Other exemplary background subtraction techniques that may be utilizedin the present invention include a variance filter, edge detection basedtechniques, or keying off skin tones using color data.

Another exemplary embodiment of the filtering step that may be used tofilter the skin 12 from the selected image 19 is band-pass filtering.One exemplary approach is illustrated in FIG. 2 which shows a print ofthe digital image after the original selected image 19 has been filteredusing a band-pass filter. The band-pass filter may comprise any suitablefilter as known by those of ordinary skill in the art. The band-passfiltering can be accomplished by low-pass filtering the selected imagetwice and then subtracting the two resulting filtered images. Theband-pass filter may comprise a first filtering step using a low-passfilter having a first kernel and a second filtering step using alow-pass filter having a second kernel. The first kernel is preferablydifferent from the second kernel. In one embodiment of the presentinvention, the kernels of the low-pass filter(s) may be Gaussiankernels. The first Gaussian kernel may have substantially the followingcharacteristics: support 21 pixels, sigma of 1.0. The second Gaussiankernel may have substantially the following characteristics: support 21pixels, sigma of 0.075.

Next, after removal of skin 12 (which may include removal of other smallbackground components as mentioned above), the resulted image isprocessed using well-known image processing techniques to produce asegmented image. One example of a segmented image is a binary image.FIG. 3 is a print of the binary image after the image has beensegmented. As discussed above, the segmented image is not limited tocreating a binary image, but may also be multi-modal, i.e.differentiated into more than two different image codes corresponding tofeatures such as moles, blood, hair and/or other features. The segmentedimage provides an improved and more clear image of the FU, however, itis still likely to contain certain “noise” that requires furtherfiltering (e.g.: objects and artifacts that do not correspond to hair orspecified hair). It was observed that removing background components, asdescribed above, prior to obtaining the segmented image containing FUnot only reduced the amount of the remaining noise in the segmentedimage, but also simplified and made more efficient the remaining stepsof filtering all other objects that do not correspond to hair ofinterest.

All noise still remaining in the segmented image could be filteredsimultaneously, or it could be broken into several separate steps: forexample, first “small noise” corresponding to the small objects (forexample, some remaining smaller blood speckles, or remaining largerpieces of dandruff) may be filtered, and then “larger noise”corresponding to the larger objects (such as long uncut hair, orpreviously cut hair that still remains on the scalp) may be filtered, orvisa versa. The example of the cut hair follicle that remains on thescalp and appears in the image would not be considered “an FU ofinterest,” therefore; it needs to be filtered as noise. Filtering of the“small noise” from the segmented image is referred to here as amorphological open operation. A morphological open operation may beaccomplished by using standard image processing technique known by thoseof ordinary skill in the art. FIG. 4 shows the resulting exemplary imageafter the morphological open operation. As can be seen in FIG. 4, theimage may still contain some objects which do not correspond to the hair13 of follicular unit 14. There are still objects which appear to be toolong, too large, randomly oriented and/or in a location which probablydoes not contain hair.

Accordingly, additional noise filtering may be performed on the imageresulting from the morphological open operation. The additional noisefiltering removes larger objects which do not meet criteriacorresponding to a follicular unit 14. Referring back to FIG. 4, theobject 22 appears to be much longer and have a much larger area than theother objects in the image 19. Thus, it can be assumed that this objectis probably not a hair 13 and therefore should be filtered out of theimage. Turning now to the print of the image after the noise filteringstep of FIG. 5, it can be seen that the object 22 has been filtered outof the image. The noise filtering step can filter based on a wide rangeof characteristics of the objects in the image, including withoutlimitation, length, area, orientation and/or location. Whether thecharacteristics of an image of an object corresponds to hair may bedetermined by statistical comparison to the global nature of the samecharacteristics for images of objects in the selected image which areknown to be hair, or alternatively, the characteristics can be comparedto predetermined criteria based on patient sampling or other data. Forinstance, the noise filtering filter can be based on characteristics ofa sampling of the other hairs on the body surface of the particularpatient, or the characteristics of a sampling of hairs on a sample ofpatients, or on known predetermined data based on studies or research.

To summarize, all noise filtering from the segmented image could beaccomplished simultaneously or in steps based on various criteria and asneeded. Each of the objects remaining in the image after the noisefiltering is counted as a follicular unit of interest. Thus, the methodmay be used to count follicular units.

The basic steps of the above-described exemplary method of countingfollicular units are summarized in the flow chart of FIG. 6. FIG. 6 issimply a flow chart representation of the method described above. Atstep 100, an image containing follicular units is acquired. At step 110,the acquired image is filtered to remove background components asexplained above. At step 120, the image without background components isprocessed to produce a segmented image. At step 130, a noise filteringprocedure is performed to remove artifacts and objects not correspondingto hairs of interest. As the result of the above steps, the number ofremaining objects may be counted to determine the number of follicularunits in the image (or a selected sub-area of the image). The FUs mayalso be labeled, if desired. FIG. 6 shows the additional counting andlabeling steps 140.

In another embodiment of the present invention, the method of countingfollicular units may be used in conjunction with a method of classifyingfollicular units, such as the methods described in U.S. patentapplication Ser. No. 11/467,268. In this way the follicular units ofinterest may be both counted and classified. The method of classifying afollicular unit (FU) comprises acquiring an image of a body surface inwhich there are follicular units (FU) and processing such image toproduce a segmented image of the FU. In one preferred embodiment thesegmented image is a binary image, but it could be a multi-modal image,as described above. From the segmented image of the FU, a contour aroundthe outer perimeter of the hair(s) of the FU may be calculated. Forexample, for an F1, the contour would generally be a line or surfacefollowing the outer surface of the single hair. For a relativelystraight hair, the contour would look like a rectangle. For an F2, thehairs typically form a “V” shape such that the contour looks like ablock lettered “V”.

The segmented image also allows the calculation of an outline profile ofthe FU. The outline profile disregards concavities in the contour of theimage. For instance, for an F2, there is a concavity or “inwardlycurved” portion in the contour formed by the descent in the contour fromthe one side of the top of the “V” to the vertex of the “V” and back upto the other side of the top of the “V”. The calculated profiledisregards this concavity such that the resulting outline profile lookslike a triangle with one of the vertices of the triangle generallytracing the vertex of the “V” of the contour of the FU.

The outline profile is then compared to the contour to determine thenumber of “defects” in the outline profile. A defect in the outlineprofile may be defined, for example, as each of the concavities in theoutline profile which divert from the contour. In the F2 example, thereis one defect in the outline profile represented by the concavity formedby the “V” shape. In an F3, the contour will be generally shaped liketwo Vs sharing a common vertex and with one line forming one side ofboth Vs. The outline profile of an F3 will also have a generallytriangular shape (although it may be a wider triangle than an F2). Thus,an F3 will have two defects. Therefore, it can be seen that the numberof defects has a direct relationship to the type of follicular unit. Inthis case, the number of hairs for the FU equals the number of defectsminus one.

In one non-limiting exemplary embodiment of the method of classifyingfollicular units, the outline profile may be determined by calculating aconvex hull contour pursuant to well-known image processing techniques.Other appropriate techniques for determining the outline profile arealso within the scope of the invention disclosed.

In another non-limiting exemplary embodiment of the method ofclassifying follicular units, procedure is provided for tracking the FUof interest to adjust for relative movement between an image acquisitiondevice and the FU. Two or more cameras, or other image acquisitiondeices, could be aligned to obtain an image or multiple images. In oneexemplary embodiment, stereo images may be used to track an FU ofinterest within the images of the first and second cameras to adjust formovement of the body surface and/or movement of the cameras. Inaddition, the first and second cameras are aligned with the generalorientation of the hair of the FU. In this way, an image is obtainedwhich provides good quality data for performing the remaining steps ofthe method of classifying the FU. The above description is provided byway of example and not limitation. Therefore, it is not necessary to usetwo cameras or stereo imaging, and the tracking procedure could beperformed with multiple image acquisition devices, such as multiplecameras, as well as with a single camera that could take multiple imagesfrom various angles, including panoramic images. Camera may be movedeither manually or with the assistance of a robot if the system used isa robotic system.

The method of classifying a follicular unit may also adjust forfollicular units having hairs which converge below the surface of theskin. In such case, the image will contain an image of a hair which isnot a contiguous part of the contour of the FU of interest. To accountfor this situation, it is determined whether the separate hair is withina maximum distance from the hair(s) defining the contiguous contour ofthe FU of interest. The maximum distance is set to be a distance inwhich what appears to be a hair from a separate FU is most likely a partof the same FU as the FU of interest. The classification of the FU ofinterest then takes into account any additional hair(s) which are withina maximum distance from the hair(s) of the FU of interest.

The method of classifying a follicular unit may also adjust for hairimages which appear to be a single hair but are in actuality multiplehairs. If the image is taken at a certain angle to the hairs of an FU,the image of the hairs may merge and appear to be one hair. Thus,determining the number of defects will not provide an accurateclassification because the merged hairs will result in fewer defects inthe outline profile (and therefore fewer hairs) than are actuallypresent in the FU of interest. To account for this situation, the methoddetermines the width (or caliber) of each object representing a hair inthe FU of interest using the image. Then, it is determined whether thewidth of each object representing a hair exceeds a maximum expectedwidth for a single hair and comparing them. The step of classifying theFU then is also based on a determination whether the width of an objectrepresenting a hair exceeds the maximum expected width and by how much.For example, if the width is between 1½ and 2 times the expected width,then the step of classifying will approximate such object as being twohairs. A similar approximation can be done for 3, 4 or 5 hairs.

It should be understood that the method of counting FUs may be performedbefore, after or simultaneously with the method of classifying.

In yet another aspect of the present invention, a system for countingfollicular units (and classifying FUs, as the case may be) is provided.As an exemplary embodiment, the system may comprise an image acquisitiondevice and an image processor. Some non-limiting examples of an imageacquisition device include one or more cameras, such as any commerciallyavailable cameras. The image acquisition device may take still images,or it could be a video recording device (such as a camcorder) or anyother image acquisition device. Stereo imaging devices are currentlypreferred, but it is not necessary to have stereo imaging and thepresent invention is not so limited. Likewise, although it is preferredthat the image acquisition device be a digital device, it is notnecessary. For example, the image acquisition device could be an analogTV camera that acquires an initial image which is then processed into adigital image for further use in the method of the present invention.The image processor used in the above system may comprise any deviceprogrammed and configured to perform the method of counting (and,optionally, classifying) a FU according to the present invention. By wayof example, and not limitation, a suitable image processor may be anytype of personal computer (“PC”). Alternatively, the image processor maycomprise an Application Specific Integrated Circuit (ASIC) or FieldProgrammable Gate Array (FPGA). In one exemplary embodiment, a systemfor counting follicular units (FUs) comprises: an image acquisitiondevice and an image processor configured for filtering the imagecomprising skin and FUs obtained from the image acquisition device toremove background components; processing the image to produce asegmented image; and performing noise filtering of the segmented imageto remove objects that do not correspond to FUs of interest.

According to another aspect of the present invention, an image processorfor counting follicular units is provided. It may comprise anyappropriate device as described above. The image processor may beprogrammed with software configured to perform the method of counting(and, optionally, classifying) follicular units. In one exemplaryembodiment, an image processor for counting follicular units isconfigured for receiving an image of a body surface comprising skin andfollicular units, filtering the image to remove background components,processing the image to produce a segmented image, and performing noisefiltering to remove objects that do not correspond to follicular unitsof interest. Such image processor according to the present inventioncould be provided separately and used in conjunction with planning ofhair treatments, or with various systems for harvesting and/orimplanting follicular units (manual, semiautomatic, automatic, orrobotic), as well as with various systems for counting or classifyingfollicular units; alternatively, it could be incorporated in any of theabove systems and devices.

The image acquisition device may be provided independently, or it may bemounted in a fixed position, or it may be mounted to a robotic arm orother controllable motion device. The robotic arm or motion device maybe operatively coupled to a controller configured to control the motionof the robotic arm or motion device. The controller may receive andprocess images or data from the image processor with the controllerconfigured to control the motion of the robotic arm or motion devicebased on the images or data acquired by the image acquisition device. Inaddition, the system may comprise a hair harvesting and/or implantationtools.

Any of the systems and methods for counting (and classifying) afollicular unit as described herein may be used in conjunction with thesystem and method of harvesting and transplanting hair as described inU.S. patent application Ser. No. 11/380,903 and U.S. patent applicationSer. No. 11/380,907.

The foregoing illustrated and described embodiments of the invention aresusceptible to various modifications and alternative forms, and itshould be understood that the invention generally, as well as thespecific embodiments described herein, are not limited to the particularforms or methods disclosed, but to the contrary cover all modifications,equivalents and alternatives falling within the scope of the appendedclaims. By way of non-limiting example, it will be appreciated by thoseskilled in the art that the invention is not limited to the use of arobotic system including a robotic arm, and that other automated andsemi-automated systems may be utilized. Moreover, the system and methodof counting follicular units of the present invention can be a separatesystem used along with a separate automated transplantation system oreven with a manual transplantation procedure.

1-31. (canceled)
 32. A method of counting follicular units (FUs),comprising: ; filtering an image of a body surface comprising skin andFUs to remove background components; processing the image to produce asegmented image; performing noise filtering of the segmented image toremove objects having characteristics that do not correspond to FUs ofinterest; and counting the FUs of interest.
 33. The method of claim 32,wherein the image is a digital image.
 34. The method of claim 32,wherein filtering to remove background components is accomplished usinga background subtraction technique.
 35. The method of claim 34, whereinthe background subtraction technique comprises computing a blurred imageand subtracting the blurred image from the image of a body surfacecomprising skin and FUs.
 36. The method of claim 35, wherein the blurredimage is computed by convolving the image of a body surface comprisingskin and FUs with a Gaussian kernel.
 37. The method of claim 36, whereinthe Gaussian kernel is tuned for one or more of a particular patient, anambient lighting condition, and other clinical condition.
 38. The methodof claim 34, wherein the background subtraction technique comprises avariance filtering.
 39. The method of claim 34, wherein the backgroundsubtraction technique comprises an edge detection based process.
 40. Themethod of claim 32, wherein filtering to remove background components isaccomplished using a band-pass filter.
 41. The method of claim 32,wherein filtering to remove background components comprises removingcomponents corresponding to the skin and other small objects from theacquired image.
 42. The method of claim 32, wherein filtering to removebackground components comprises low-pass filtering a selected imagetwice and then subtracting the two resulting filtered images from theimage of a body surface comprising skin and FUs.
 43. The method of claim32, wherein filtering to remove background components comprises a firstfilter step in which the image is filtered using a low-pass filterhaving a first kernel, and a second filter step in which the image isfiltered using a low-pass filter having a second kernel.
 44. The methodof claim 43, wherein the first kernel is a Gaussian kernel havingsubstantially the following characteristics: support 21 pixels, andsigma of 1.0.
 45. The method of claim 43, wherein the second kernel is aGaussian kernel having substantially the following characteristics:support 21 pixels, and sigma of 0.75.
 46. The method of claim 43,wherein the first kernel is different from the second kernel.
 47. Themethod of claim 32, further comprising: acquiring at least oneadditional image of a same field of view at a known position differentfrom a vantage point of the first image of a body surface comprisingskin and FUs, wherein the first image and the at least one additionalimage are acquired in sequence; computing the coordinate position of ahair on the body surface using the first and the at least one additionalimages; and filtering out follicular units having a computed coordinateposition which is inconsistent with a hair on the body surface.
 48. Themethod of claim 32, wherein noise filtering comprises filtering out anyobject having an area that differs from a mean object size within theimage of a body surface comprising skin and FUs.
 49. The method of claim32, wherein noise filtering comprises filtering out objects whosecharacteristics include one or more of area, length, location andorientation.
 50. The method of claim 32, wherein noise filteringcomprises filtering out objects whose characteristics do not correspondto one or more of characteristics for hair based on a sampling of hairson the body surface and characteristics expected for hair based onpredetermined data.
 51. The method of claim 32, further comprisingtracking an FU being counted by: acquiring a plurality of images of theFU in sequence; determining positions of the FU from each of theplurality of images; tracking movement of the FU based upon thedetermined positions.
 52. The method of claim 32, further comprisingclassifying an FU based on a number of hairs emanating from the FU. 53.The method of claim 32, further comprising calculating a contour of thesegmented image of a FU to be classified; calculating an outline profileof the segmented image which disregards concavities in the contour ofthe segmented image of the FU to be classified; determining a number ofdefects in the outline profile of the FU to be classified; andclassifying the FU at least partially based on the number of determineddefects.
 54. The method of claim 53, wherein the FUs of interest arecounted based at least in part upon results of classifying one or moreFUs to account for any multiple FUs that appear to be a single FU,and/or any single FUs that appear to be multiple FUs.
 55. A system forcounting follicular units (FUs) on a body surface, comprising: an imageacquisition device; and an image processor comprising instructions forfiltering an image comprising skin and FUs obtained from the imageacquisition device to remove background components; processing the imageto produce a segmented image; performing noise filtering of thesegmented image to remove objects having characteristics that do notcorrespond to FUs of interest; and counting the FUs of interest.
 56. Thesystem of claim 55, wherein the system is a robotic system.
 57. Thesystem of claim 56, further comprising a robotic arm on which the imageacquisition device is mounted.
 58. The system of claim 57, furthercomprising a controller operatively coupled to the robotic arm and theimage processor.
 59. The system of claim 55, wherein the image processoris a personal computer.
 60. The system of claim 55, wherein the imageacquisition device comprises one or more cameras. 61-62. (canceled) 63.A system for counting follicular units (FUs) on a body surface,comprising: an interface adapted to receive image data containing FUs;and an image processor comprising one or more modules for executingoperations on the image data, the one or more modules includinginstructions for receiving an image of a body surface comprising skinand FUs; filtering the image to remove background components; processingthe image to produce a segmented image; and performing noise filteringto remove objects having characteristics that do not correspond to FUsof interest.
 64. The system of claim 63, wherein the image processor isprogrammed for classifying FUs based on a number of hairs emanating fromthem.